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Department  of  Psychology  Campus  address: 

RO.  Box  11 A  Yale  Station  2  Hillhouse  Avenue. 

New  Haven,  Connecticut  06320-7447 


September  16, 1995 


Defense  Technical  Information  Center 
Building  5,  Cameron  Station 
Alexandria,  VA  22304-6145 


Re:  Final  Technial  Report  on  ONR  grant  NOOO 14-92- 192: 


Dear  Sirs/Mesdames: 

This  is  the  final  technical  report  on  grant  N00014-92-J-1923.  This  information  has 
also  been  sent  to  Dr.  Thomas  McKenna,  the  Administrative  Grants  Officer,  and  the 
Director  of  the  Naval  Research  Laboratory. 

The  overarching  goal  of  this  grant  "Neuronal  Micronets  as  Nodal  Elements" 
(N00014-92-J-1923)  application  was  to  determine  the  computational  significance  of  the 
amount  and  type  of  information  processing  that  actually  occurs  in  the  nervous  system  at  the 
single-neuron  level. 

The  basic  thesis  was  that  a  neuron  can  be  compared  to  a  multi-layered  artificial 
neural  network  (ANN).  The  question  is,  what  kind  of  ANN  best  captures  neuronal 
computations?  TTiis  question  raises  several  others:  How  do  neurons  differ  from  the 
processing  elements  (PEs)  used  in  connectionistic  studies?  What  do  single  neurons 
compute?  We  have  been  addressing  the  latter  question  through  compartmental  simulations 
of  hippocampal  neurons  containing  Hebbian  synapses. 

I  used  the  term  "micronet”  to  refer  to  the  type  of  ANN  that  can  capture  neuronal 
computations  1.  I  concluded  that  micronets  are  deep  (many  layers  of  PEs)  but  narrow 
(relatively  few  PEs  per  layer).  To  function  like  a  neuron,  the  PEs  must  operate 
continuously  and  asynchronously.  There  is  no  clock.  Time  is  its  own  representation. 
Connections  within  a  micronet  are  assumed  not  to  be  modifiable,  but  connections  among 
micronets  can  exhibit  use-dependent  modifications,  which  can  be  Hebbian.  The  PE 
activation  function  has  a  passive  memory  that  decays  rapidly  and  exponentially  as  a 
function  of  time. 

I  requested  an  AASERT  for  Sean  Murphy,  a  neuroscience  graduate  student 
interested  in  this  problem,  but  this  application  was  not  funded.  Murphy  went  on  however, 
working  with  another  professor  in  my  department,  to  begin  building  ANNs  that  could 
emulate  the  input-output  functions  of  neurons,  and  this  grew  into  part  of  his  dissertation, 
which  was  just  completed^.  As  I  had  anticipated,  Murphy  concluded  that  an  appropriate 
neural  network  can  indeed  emulate  a  relatively  realistic  neuronal  model  in  terms  of  gross 
input-output  functions. 

In  working  with  one  of  my  former  students,  we  reached  a  similar  conclusion,  based 
on  less  formal  or  extensive  analysis.  The  main  difference,  however,  was  that  we 
concluded  that,  once  learning  is  involved,  the  circumstance  is  quite  different.  And  learning 
was  an  essential  part  of  the  problem  we  sought  to  understand.  What  follows  summarizes 
some  of  the  reasons  for  rejecting  our  initial  basic  thesis  that  a  neuron  can  be  compared  to  a 
multilayered  ANN.  The  point  is  not  that  the  enterprise  cannot  be  done  in  principle,  but  that 
this  approach  is  much  too  cumbersome  relative  to  certain  alternatives,  if  one  is  interested  in 
learning. 
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Part  of  the  reason  is  that  Hebbian  learning  depends  on  the  electrotonic  structure  of 
the  cells  and  this  is  not  easy  to  capture  in  connectionist  models^-^.  Hebbian  learning 
depends  on  the  amplitude  of  the  signal  at  the  site  of  the  synaptic  input.  This  in  turn  means 
that  voltage  transfers  to  and  from  that  site  to  and  from  every  other  synapse  must  be 
modeled.  In  addition,  the  effects  of  spiking  in  the  soma  must  be  represented  in  terms  of 
voltage  transfer  back  through  the  dendritic  tree  to  the  synapses. 

These  voltage  transfers  are  in  general  not  symmetrical — the  attenuation  from  point  i 
to  point  j  is  not  the  same  as  from  j  to  i  and  they  are  frequency  dependent!®.  Furthermore, 
nonlinear  membrane  responses,  which  our  modeling  suggested  to  be  important! 
particularly  the  backpropagation  of  action  potentials  into  the  dendrites^,  are  very  difficult  to 
incorporate  in  an  ANN  that  includes  Hebbian  learning  but  have  recently  been  suggested 
based  on  experimental  data. 

Why  this  insistence  on  Hebbian  learning?  Based  on  first  principles,  it  has  always 
been  clear  that  Hebbian  synapses  were  theoretically  important^>!3,14  and  we  and  others  had 
previously  shown  them  to  exist  in  hippocampus.  Now  it  seems  that  Hebbian  synapses  are 
extremely  widespread.  One  sees  such  synapses  in  brain  regions  other  than  the 
hippocampus.  One  even  sees  such  synapses  in  lower  vertebrates.  Most  recently,  it  seems 
that  they  may  even  exist  in  invertebrates. 

Thus  we  continued  to  explore  electrotonic  structure  in  order  to  understand  better 
how  it  might  interact  with  active  membrane  and  Hebbian  leaming!^"^®.  At  the  same  time, 
we  developed  advanced  methods  to  gain  a  better  understanding  of  the  diversity  of  cell  types 
and  their  characteristic  active  membrane  properties!^’^!. 

My  conclusion  from  this  work  was  that  it  is  computationally  easier  to  use  analog 
models  or  devices  instead  of  trying  to  make  m  ANN  emulate  what  such  circuitry  would  do. 
If  learning  can  be  done  off  line,  or  if  learning  is  not  involved,  then  the  original  idea  of 
representing  a  neuron  as  a  micronet  still  makes  sense^.  But  from  what  we  now  know 
about  the  neurophysiology,  the  nervous  system  is  continuously  self-organizing  and 
exhibiting  various  forms  of  learning  and  that  these  will  ultimately  depend  critically  on  the 
electrotonic  structure  and  nonlinear  dynamics.  Therefore,  I  focused  on  the  latter.  We  have 
gained  deep  insights  into  single  neuron  computations  from  this  enterprise. 

We  have  now  begun  to  formalize  the  manner  in  which  Hebbian  self-organization 
depends  on  electrotonic  structure!®>^^.  The  effect  of  nonlinear  membrane  probably  will 
differ  in  different  classes  of  cells  and  for  different  types  of  synaptic  inputs.  What  we  are 
learning  is  that  there  is  no  canonical  neuron,  even  within  highly  circumscribed  brain 
regions.  The  number  of  different  types  is  large  and  we  suspect  that  this  means  that  brain- 
style  computations  require  a  correspondingly  large  number  of  elemental  devices — in 
contrast  to  the  conventional  assumptions  in  AbWs. 

You  will  recall  that  this  application  was  submitted  in  March  of  1992,  and  I  received 
notice  that  it  would  receive  no  further  funding  on  August  of  1993.  At  this  time  I  requested 
a  no-cost  extension. 

We  are  still  publishing  work  from  this  period  and  the  enterprise  motivated  us  to 
collect  experimental  data  that  confirmed  the  conclusions  based  on  the  simulations 
mentioned  above.  Additional  manuscripts  are  in  preparation.  I  am  currently  working  on 
what  I  think  may  be  a  fundamental  new  mechanism  for  how  the  nervous  system  encodes 
time  using  Hebbian  synapses.  I  am  also  continuing  to  work  on  the  design  of  a  simple 
device  that  can,  with  small  quantitative  variations,  learn  and  self-organize  the  way  real 
neurons  do.  In  contrast  to  Murphy’s  approach^,  this  elemental  device  would  not  be  an 
ANN  that  emulates  a  particular  neuron.  Rather  I  am  looking  for  an  analog  device  that  can 
capture  the  essence  of  our  insights  based  on  the  above  considerations.  I  hope  to  explore 
these  ideas  further  through  interactions  with  certain  electrical  engineers  at  Yale. 
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